Apparatus and method for three-dimensional image capture with extended depth of field

ABSTRACT

An optical system for capturing three-dimensional images of a three-dimensional object is provided. The optical system includes a projector for structured illumination of the object. The projector includes a light source, a grid mask positioned between the light source and the object for structured illumination of the object, and a first Wavefront Coding (WFC) element having a phase modulating mask positioned between the grid mask and the object to receive patterned light from the light source through the grid mask. The first WFC element is constructed and arranged such that a point spread function of the projector is substantially invariant over a wider range of depth of field of the grid mask than a point spread function of the projector without the first WFC element.

CROSS-REFERENCE TO RELATED APPLICATIONS

U.S. Pat. No. 5,748,371, issued May 5, 1998 and entitled “Extended Depthof Field Optical Systems,” is incorporated herein by reference. U.S.Pat. No. 6,069,738, issued on May 30, 2000, and entitled “Apparatus andMethods for Extending Depth of Field in Image Projection Systems,” isincorporated herein by reference. U.S. Pat. No. 6,525,302, issued onFeb. 25, 2003, and entitled “Wavefront Coding Phase Contrast ImagingSystems,” is incorporated herein by reference. International PatentApplication PCT/US2006/036556, filed Sep. 19, 2006 and entitled“Task-Based Imaging Systems”, is incorporated herein by reference.

BACKGROUND

Structured illumination (SI) or patterned illumination projects a narrowband of light onto a three-dimensional scene to produce lines ofillumination that appear distorted. The distortions can be imaged andused to reconstruct the surface shape of one or more objects within thescene, by triangulation of the position of the distorted lines.

Patterns of parallel stripes are widely used in structured illumination.Two common methods for stripe pattern generation are laser interferenceand projection. The laser interference method uses two laser beams whichinterfere to generate regular line patterns. Different pattern sizes canbe obtained by changing the angle between these beams. This methodgenerates fine patterns with unlimited depth of field. However, thelaser interference technique has disadvantages including high costassociated with implementation, incapability of modulating individualstripes, and possible interference with beams reflected from objects.

The projection method, on the other hand, uses a projector with anincoherent light source for generating patterned light (e.g. a videoprojector). Patterns may be generated by a display within the projector,such as a liquid crystal (LCD) display.

There still remains a need for developing systems and methods forcapturing three-dimensional images with high resolution and low cost.Furthermore, there is a need for a robust method to detect the projectedlines over a wide depth of field (DOF).

SUMMARY

This disclosure advances the art by providing a three-dimensional imagecapture system with extended depth of field. The three-dimensional imagecapture system incorporates Wavefront Coding (WFC) elements to extendthe depth of field.

In an embodiment, an optical system for capturing three-dimensionalimages of a three-dimensional object is provided. The optical systemincludes a projector for structured illumination of the object. Theprojector includes a light source, a grid mask positioned between thelight source and the object for structured illumination of the object,and a first Wavefront Coding (WFC) element having a phase modulatingmask positioned between the grid mask and the object to receivepatterned light from the light source through the grid mask. The firstWFC element is constructed and arranged such that a point spreadfunction of the projector is substantially invariant over a wider rangeof depth of field of the grid mask than a point spread function of theprojector without the first WFC element.

In another embodiment, a folded optical system for capturing images of athree-dimensional object is provided. The optical system includes aprojector for structured illumination of the object. The projectorcomprises includes a light source, a grid mask positioned between thelight source and the object for structured illumination of the object, afirst Wavefront Coding (WFC) element having a phase modulating maskpositioned between the grid mask and the object, and a beam splitterbetween the first WFC element and the object for changing a lightdirection from the light source. The first WFC element is constructedand arranged such that a point spread function of the projector is lesssensitive to a depth of field of the grid mask than a point spreadfunction of the projector without the first WFC element.

In a further embodiment, an optical system for capturing images of athree-dimensional object is provided. The optical system comprising aprojector for structured illumination of the object. The projectorincludes a light source, a physical medium embossed to have a surfacerelief pattern according to a computer generated hologram (CGH). Thephysical medium is positioned between the light source and the objectfor structured illumination of the object. The CGH includes a firstcomputer representation of a grid mask and a second computerrepresentation of a first Wavefront Coding (WFC) element. The opticalsystem also includes a beam splitter between the physical medium and theobject for changing a light direction from the light source. Thephysical medium is constructed and arranged such that a point spreadfunction of the projector is less sensitive to a depth of field of thephysical medium than a point spread function of the projector withoutthe physical medium.

In an embodiment, a method for capturing an image of a three-dimensionalobject is provided. The method includes (1) projecting light from aprojector through a grid mask and a phase modulating mask to generatepatterned light; (2) illuminating the three-dimensional object with thepatterned light; and (3) capturing at a detector patterned lightreflected by the three-dimensional object.

In an embodiment, a method for capturing an image of a three-dimensionalobject is provided. The method includes projecting patterned light froma projector toward the three-dimensional object, where the projectorincludes a light source and a physical medium embossed to have a surfacerelief pattern according to a computer generated hologram (CGH), thephysical medium is positioned between the light source and the objectfor structured illumination of the object. The CGH includes a firstcomputer representation of a grid mask and a second computerrepresentation of a first Wavefront Coding (WFC) element. The methodalso includes illuminating the patterned light onto thethree-dimensional object. The method further includes capturing imagesof the three-dimensional object at a detector, wherein the physicalmedium is constructed and arranged such that a first point spreadfunction of the projector is less sensitive to a depth of field of thephysical medium than a point spread function of the projector withoutthe physical medium.

Additional embodiments and features are set forth in the descriptionthat follows, and will become apparent to those skilled in the art uponexamination of the specification or may be learned by the practice ofthe invention. A further understanding of the nature and advantages ofthe present invention may be realized by reference to the remainingportions of the specification.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified diagram of a conventional optical system forstructured illumination of an object.

FIG. 2 is a simplified diagram of an optical system with extended depthof field in an embodiment.

FIG. 3 is a folded optical system with extended depth of field in anembodiment.

FIG. 4 is a gray image of a grid viewed by the conventional opticalsystem of FIG. 1, at best focus.

FIG. 5 is a gray image of a grid viewed by the conventional opticalsystem of FIG. 1, at defocus of five waves.

FIG. 6 is a gray image of a grid viewed by the conventional opticalsystem of FIG. 1, at defocus of eight waves.

FIG. 7 is a gray image of a grid viewed by the extended depth of fieldoptical system of FIG. 2 or FIG. 3, at best focus.

FIG. 8 is a gray image of a grid viewed by the optical system withextended depth of field of FIG. 2 or FIG. 3, at defocus of five waves.

FIG. 9 is a gray image of a grid viewed by the optical system withextended depth of field of FIG. 2 or FIG. 3, at defocus of eight waves.

FIG. 10 is a contour plot of the grid of FIG. 4, viewed by theconventional optical system of FIG. 1, at best focus.

FIG. 11 is a contour plot of the grid of FIG. 5, viewed by theconventional optical system of FIG. 1, at defocus of five waves.

FIG. 12 is a contour plot of the grid of FIG. 6, viewed by theconventional optical system of FIG. 1, at defocus of eight waves.

FIG. 13 is a contour plot of the grid of FIG. 7, viewed by the opticalsystem with extended depth of field of FIG. 2 or FIG. 3, at best focus.

FIG. 14 is a contour plot of the grid of FIG. 8, viewed by the opticalsystem with extended depth of field of FIG. 2 or FIG. 3, at defocus offive waves.

FIG. 15 is a contour plot of the grid of FIG. 9, viewed by the opticalsystem with extended depth of field of FIG. 2 or FIG. 3, at defocus ofeight waves.

FIG. 16 is a gray image of a point (e.g. intersection of grid), viewedby the conventional optical system of FIG. 1, at best focus.

FIG. 17 is a gray image of the point of FIG. 16, viewed by theconventional optical system of FIG. 1, at defocus of five waves.

FIG. 18 is a gray image of the point of FIG. 16, viewed by theconventional optical system of FIG. 1, at defocus of eight waves.

FIG. 19 is a gray image of a point (e.g. intersection of grid), viewedby the optical system with extended depth of field of FIG. 2 or FIG. 3,at best focus.

FIG. 20 is a gray image of the point of FIG. 19, viewed by the opticalsystem with extended depth of field of FIG. 2 or FIG. 3, at defocus offive waves.

FIG. 21 is a gray image of the point of FIG. 19, viewed by the opticalsystem with extended depth of field of FIG. 2 or FIG. 3, at defocus ofeight waves.

FIG. 22 is a point spread function (PSF) of the conventional opticalsystem of FIG. 1, at best focus.

FIG. 23 is a PSF of the conventional optical system of FIG. 1, atdefocus of five waves.

FIG. 24 is a PSF of the conventional optical system of FIG. 1, atdefocus of eight waves.

FIG. 25 is a PSF of the optical system with extended depth of field ofFIG. 2 or FIG. 3, at best focus.

FIG. 26 is a PSF of the optical system with extended depth of field ofFIG. 2 or FIG. 3, at defocus of five waves.

FIG. 27 is a PSF of the optical system with extended depth of field ofFIG. 2 or FIG. 3, at defocus of eight waves.

FIG. 28 is a contour plot of the point of FIG. 16, viewed by theconventional optical system of FIG. 1, at best focus.

FIG. 29 is a contour plot of the point of FIG. 17, viewed by theconventional optical system of FIG. 1, at defocus of five waves.

FIG. 30 is a contour plot of the point of FIG. 18, viewed by theconventional optical system of FIG. 1, at defocus of eight waves.

FIG. 31 is a contour plot of the point of FIG. 19, viewed by the opticalsystem with extended depth of field of FIG. 2 or FIG. 3, at best focus.

FIG. 32 is a contour plot of the point of FIG. 20, viewed by the opticalsystem with extended depth of field of FIG. 2 or FIG. 3, at defocus offive waves.

FIG. 33 is a contour plot of the point of FIG. 21, viewed by the opticalsystem with extended depth of field of FIG. 2 or FIG. 3, at defocus ofeight waves.

FIG. 34 is a simplified diagram of optical system with extended depth offield in an alternative embodiment.

DETAILED DESCRIPTION

The present disclosure may be understood by reference to the followingdetailed description considered in conjunction with the drawings. Notethat, for purposes of illustrative clarity, certain elements in thedrawings are not drawn to scale. Reference numbers for items that appearmultiple times may be omitted for clarity. Where possible, the samereference numbers are used throughout the drawings and the followingdescription to refer to the same or similar parts.

Structured illumination may project patterned light through a grid ontoan object to capture surface slopes of the object by measuring surfacedistortion of the grid. This structured illumination technique requiresprojecting and detecting fine lines of the grid over a capture volume ofthe SI system. Thus, the capture volume of a conventional SI system islimited by the system's depth of field such that it is difficult toaccurately project and/or detect the grid lines and their crossingpoints over a large capture volume.

Wavefront Coding (“WFC”) is a method for extending the depth of field ofan optical system and for correcting optical aberration. WFC utilizes,for example, specially designed phase masks to produce a point spreadfunction with an extended depth of field or focus (“EdoF” or “EDOF”).The point spread function (“PSF”) describes the response of an imagingsystem to a point source or point object. WFC may use, for example, acubic phase mask that blurs an image uniformly. A digital imageprocessor may then remove the blur (e.g., when a viewable image isdesired). However, dynamic range may be sacrificed to extend the depthof field. That is, a modulation transfer function (which can be thoughtof as a measure of contrast) of the imaging system may be low at certainspatial frequencies in the as-detected image data. The modulationtransfer function can be increased with image processing, but amplifyinga low-contrast signal generally also amplifies noise at the same spatialfrequency.

The present disclosure provides systems and methods that integrateimaging systems with WFC to extend a depth of field to capture 3D imagesat low cost. More specifically, the systems use a first WFC element toproject, and an optional second WFC element to detect, grids ofstructured illumination over an extended depth of field. The WFCelements may also reduce chromatic aberration for color encodedstructured illumination.

FIG. 1 is a simplified diagram of a conventional optical system 100 forstructured illumination of an object. Conventional optical system 100may include a projector 102 and a detector 104 (e.g., a camera) forviewing object 106. Projector 102 includes a light source for emittinglight toward object 106. Detector 104 receives scattered or diffractedlight from object 106.

FIG. 2 is a simplified diagram of an optical system 200 with extendeddepth of field. System 200 includes a projection arm 220 that has aprojector 202 with a first WFC element 208, and a detection arm 230 thathas a detector 204 (e.g. a camera) with an optional, second WFC element214. Light from projector 202 passes through first WFC element 208 andimpinges on object 206. Light scatters or diffracts from object 206toward detector 204 through second WFC element 214. Projector 202 mayinclude a grid mask 210 for structured illumination.

First WFC element 208 provides phase modulations for grid 210 such thatgrid 210 appears to be less sharp in system 200 than in conventionalsystem 100 at best-focus. However, the incorporation of first WFCelement 208 into system 200 allows grid 210 of projector 202 to bedetected over a wider range of depth of field, such that centroids ofgrid 210 may be accurately determined over the wider range of depth offield. Although system 200 has extended depth of field, asignal-to-noise ratio (SNR) may be reduced in system 200 at best focusas a result of incorporation of first WFC element 208.

The optional, second WFC element 214 is integrated into system 200 forcompensation of lateral shifts introduced by first WFC element 208.However, second WFC element 214 may blur the image. Second WFC element214 may be movable or removable and used only when needed forcompensating lateral shifts. The first and second WFC elements arediscussed further below.

A limitation of system 200 is that object 206 should not, but might,block illumination from projector 202. In system 200, projector 202 anddetector 204 are not aligned. A projector and a detector mayadvantageously be aligned such that light from the projector impinges onan object and the light scatters back toward the detector in the samedirection as the incoming direction of the light as detectable by thesystem. The advantage of the aligned projector and detector is that allsurfaces that are illuminated by the projection optics are “visible” tothe receiving optics, and vice versa. Some objects may be behind otherobjects, but at least all the objects “visible” to the receiving opticswill be illuminated. An example is provided for such a system that has aprojector aligned with a detector.

FIG. 3 is a folded optical system 300 with extended depth of field.Folded optical system 300 includes an illumination arm or a projectionarm 320 and a detection arm 330, bounded by dashed lines. The projectionarm 320 may include a light source 302, a grid mask 304, a collimatinglens 306A, a first WFC element 308, lens 306B, a dual-use folding beamsplitter 310, and a dual-purpose lens 312. The detection arm 330includes lens 312, beam splitter 310, an optional second WFC element314, lens 322, and a detector 316. Beam splitter 310 and lens 312 areused in both projection arm 320 and detector arm 330 for a dual-purpose.

In projection arm 320, light from light source 302 is condensed throughcollimating lens 306A which may be at approximately one focal lengthfrom light source 302. Collimating lens 306A provides uniformillumination for grid mask 304. Lens 306B may be placed approximatelyone focal length from grid mask 304 and Fourier transforms an image ofgrid mask 304. A Fourier plane is a plane in space perpendicular to theoptic axis where an image is Fourier transformed by a lens. For an imageof one focal length before the lens, its Fourier transform is found onefocal length after the lens. First WFC element 308 is positioned betweenlens 306B and beam splitter 310 and near a Fourier plane 320A ofprojection arm 320, such that grid mask 304 remains well-focused over alarger distance than without use of first WFC element 308. Fourier plane320A is approximately one focal length from lens 312 and one focallength from lens 306B. Lens 312 is a dual purpose lens, which operatesas a projection lens for the projection arm 320 and as aFourier-transform lens for the detection arm 330. First WFC element 308in projection arm 320 makes grid mask 304 appear to be less sharp than agrid mask of conventional optical system 100 at best focus. Therefore,the signal-to-noise ratio of system 300 may be lower than that ofconventional optical system 100 at best focus. However, system 300allows grid mask 304 to be detected over an extended depth of field suchthat centroids of grid mask 304 may be accurately determined over theextended depth of field.

Light source 302 may emit visible light or invisible light. For example,infrared light may be used when a user of system 300 does not wantobject 318 (e.g. human or animal) to be aware of image capture. In analternative embodiment, one or more of grid mask 304, collimating lens306, and first WFC element 308 may be combined or integrated into asingle element 222 (bounded by dotted lines), resulting in reduced partcount and alignment steps required by system 300 and potentiallyreducing fabrication costs and increasing the quality and performance ofthe system. Grid mask 304 may be movable or removable such thatprojection arm 320 may be used for illuminating object 318 therebyenabling system 300 to capture simple images of object 318 as a camerawould.

Beam splitter 310 allows system 300 to be folded by changing lightdirection from projector arm 320 so that it shares an optical path withlight received by detector 316. Beam splitter 310 partially reflects anincoming beam 332 from light source 302 toward object 318. Morespecifically, beam 332 impinges at about 45 degrees on surface 334 ofbeam splitter 310, which at least partially reflects beam 332 fromsurface 334 to form beam 332 a toward object 318. Beam 332 a impinges onobject 318 and scatters back and transmits through beam splitter 310,and may pass through optional second WFC element 314, and forms an imageof illuminated object 318 on detector 316. Beam splitter 310 may also bemovable or removable to allow low-light or low-illumination operation ofsystem 300 when no illumination from projection arm 320 is necessary. Areflection surface 334 of beam splitter 310 may be, for example, ahalf-silvered mirror. Surface 334 may be made of a plate of glass with athin metallic coating (e.g. aluminum) or a dielectric coating.Reflection to transmission ratios of surface 334 may vary with materialsand wavelength.

In detection arm 330, lens 322 operates as an eyepiece lens, forming animage onto detector 316. Lens 322 is placed approximately one focallength from detector 316. Beam splitter 310 transmits light from object318 to detector 316. Second WFC element 314 may be optionally included,to provide extended depth of field for imaging of object 318, and/or tocompensate for lateral shifts generated by first WFC element 308.

In an embodiment, first WFC element 208 or 308 may include high-orderseparable (HOS) elements and high-order nearly separable (HONS) elementsor weakly non-separable elements. The HOS and/or HONS elements have thebenefit of concentrating their modulation in the horizontal and verticaldirections (the directions along which most of the grid images areoriented). However, the HOS and/or HONS elements may also introduce alateral shift of the point spread function (PSF) as a function ofdefocus, which means that points of three-dimensional object 318 atvarious depths have different lateral shifts in the image plane ofdetector arm 330.

Second WFC element 314 may be placed near a Fourier plane 330A ofdetection arm 330 to increase the DOF not only of the captured images ofobject 318 but also of the captured images of projected grid mask 304.Fourier plane 330A is approximately one focal length from lens 322 andapproximately one focal length away from lens 312. Second WFC element314 may be the same type as first WFC element 208 or 308, but rotated soas to compensate or correct the lateral shifts caused by first WFCelement 308, as now explained.

Assuming beam splitter 310 is used, to combine both projection anddetection arms 320 and 330 into one system 300, and beam splitterincludes reflection surface 334 at a forty-five degree angle, there willbe a rotation of 90 degrees between an optical axis 350 of projectionarm 320, including first WFC element 308, and an optical axis 360 ofdetection arm 330, including second WFC element 314. For example,optical axis 350 of projection aim 320 is substantially perpendicular tooptical axis 360 of detector arm 330. First WFC element 308 may have aphase profile in x and y coordinates and optical axis 350 along the zaxis direction, according to coordinate system 370A shown in FIG. 3.Second WFC element 314 has a phase profile in X′ and Y′ coordinates, andoptical axis 360 along Z′ axis, according to coordinate system 370Bshown in FIG. 3.

As illustrated in FIG. 3, the Y′ axis of second WFC element 314 is inthe same direction as the y axis of first WFC element 308. The Z′ axisof second WFC element 314 rotates counter clockwise around the y axis offirst WFC element 308 for 90 degrees such that the Z′ axis of second WFCelement 314 is in the same direction as that of the x axis of first WFCelement 308. The X′ axis of second WFC element 314 is opposite to the zaxis of first WFC element 308 such that lateral shifts generated byfirst WFC element 308 may be compensated by second WFC element 314.First WFC element 308 has two ends 340A and 340B, where end 340A is at amore positive x coordinate than end 340B. For the orientation of secondWFC element 314 relative to first WFC element 308, one may imagine thatsecond WFC element 314 may rotate counter-clockwise by 90 degrees aboutthe y axis of first WFC element 308 from end 340A of first WFC element308. After the 90 degrees rotation, second WFC element 314 flips 180degrees around the Y′ axis of second WFC element 314. One end 355B ofsecond WFC element 314 is at a more positive X′ coordinate than a secondend 355A of second WFC element 314 after the rotation. Ends 355A and355B of second WFC elements correspond to ends 340A and 340Brespectively. This way all points between grid mask 304 and detector 316experience approximately the same amount of phase modulation throughfirst and second WFC elements 308 and 314, thus compensating for anylateral shift introduced by first WFC element 308 as a function ofdefocus.

By using a cubic or a HOS function in first WFC element 208 or 308, mostmodulations are concentrated in the x and y directions where grid linesare undistorted, for example, 60%, 80%, 90% or higher. In other words,first WFC element 208 or 308 is configured to concentrate most phasemodulations along grid lines (e.g. grid lines 702, 802 and 902 of FIGS.7-9).

In an embodiment, the first and second WFC elements may include a weaklynon-separable function. The weakly non-separable function of the firstWFC element may be expressed as:

$\begin{matrix}{{P\left( {x,y} \right)} = {{\sum\limits_{i}^{N}{\sum\limits_{j}^{N}{\alpha_{i}x^{i}}}} + {\beta_{j}y^{j}} + {\gamma\; x^{i}y^{j}}}} & {{Equation}\mspace{14mu}(1)}\end{matrix}$where i and j are positive integers starting from 1, and N is thehighest polynomial order. In practice, N is usually limited to about 5,since the effect of higher order terms are usually lost due to thepractical limits of fabrication tolerances. The weakly non-separablefunction of second WFC element 314 may be expressed as:

$\begin{matrix}{{P\left( {X^{\prime},Y^{\prime}} \right)} = {{\sum\limits_{i}^{N}{\sum\limits_{j}^{N}{\alpha_{i}\left( {- X^{\prime}} \right)}^{i}}} + {\beta_{j}\left( {- Y^{\prime}} \right)}^{j} + {{\gamma\left( {- X^{\prime}} \right)}^{i}\left( {- Y^{\prime}} \right)^{j}}}} & {{Equation}\mspace{14mu}(2)}\end{matrix}$where i and j are positive integers starting from 1, and where the minussigns represent the 180 degrees rotation about optic axis 360 of secondWFC element 314, as previously described. Coefficients α and β may ormay not be equal. When coefficient γ is equal to zero, the weaklynon-separable function becomes a high order separable function. FirstWFC element 308 may include a high order separable function expressedas:

$\begin{matrix}{{P\left( {x,y} \right)} = {{\sum\limits_{i}^{N}{\sum\limits_{j}^{N}{\alpha_{i}x^{i}}}} + {\beta_{j}y^{j}}}} & {{Equation}\mspace{14mu}(3)}\end{matrix}$Second WFC element 314 may include a high order separable functionexpressed as:

$\begin{matrix}{{P\left( {X^{\prime},Y^{\prime}} \right)} = {{\sum\limits_{i}^{N}{\sum\limits_{j}^{N}{\alpha_{i}\left( {- X^{\prime}} \right)}^{i}}} + {\beta_{j}\left( {- Y^{\prime}} \right)}^{j}}} & {{Equation}\mspace{14mu}(4)}\end{matrix}$In a particular embodiment, the high order separable function of firstWFC element 308 may be a cubic function expressed as:P(x,y)=αx ³ +βy ³  Equation (5)The cubic function of second WFC element 314 is expressed as:P(x,y)=−αX′ ³ +βY′ ³)  Equation (6)Coefficients α and β may or may not be equal.

In an alternative embodiment, second WFC element 214 or 314 may bedifferent from first WFC element 214 or 314. For example, second WFCelement 214 or 314 may have a circularly symmetric function. Thecircularly symmetric function does not compensate for the lateral shiftsgenerated by first WFC element 208 or 308, and further blurs the imageof the object. However, the circularly symmetric function promotesproduction of images that are relatively natural looking and may haverelatively few artifacts. Accordingly, the circularly symmetric functionmay be well suited for applications when detection arm 330 is used tocapture images that are pleasing to a human viewer. The circularlysymmetric function may be expressed as:P(ρ)=f(ρ)  Equation (7)where ρ²=x²+y² and ρ is the radius of the Fourier plane 330A of detectorarm 330, and f(ρ) is a high order polynomial.

Second WFC element 214 or 314 may be movable or removable for use incapturing scenes that do not need extended depth of field (e.g. whenusing system 200 or 300 as a camera).

In an alternative embodiment, first WFC element 208 or 308 may becircularly symmetric such that the projected grid lines advantageouslydo not suffer from focus-dependent lateral shift. However, modulation isundesirably spread evenly over all directions and not concentrated inthe directions of the undistorted grid lines. In this configuration,second WFC element 214 or 314 may also be circularly symmetric for EDOF.As discussed above, second WFC element 214 or 314 are movable orremovable in certain embodiments.

FIGS. 4-33 include simulated gray images of a grid and a single point atfocus and defocus, contour plots of the grid and single point at focusand defocus, and simulated PSFs in three-dimensional mesh drawings atfocus and at various amounts of defocus. These figures are provided forboth conventional system 100 without a WFC element, and for opticalsystem 200 or 300 with extended depth of field, that is, utilizing anexemplary first WFC element 208 or 308. Potential advantages of system200 or 300 over the conventional system 100 are demonstrated insubsequent paragraphs below.

FIGS. 4-6 are gray images of a grid viewed by conventional opticalsystem 100 of FIG. 1 at best focus or zero wave of defocus, five wavesof defocus, and eight waves of defocus respectively, where a wave isdefined as the center wavelength of illuminating light from a lightsource (e.g. from projector 102 in FIG. 1). Referring to FIG. 4 now,gray image 400 at best focus clearly shows a grid. Gray image 500 ofFIG. 5 at five waves of defocus is blurred as compared to gray image400. Referring to FIG. 6 now, gray image 600 at eight waves of defocusis even more blurred than gray image 500. Grid lines can not be seenclearly in gray images 500 and 600. Images 400, 500 and 600 arenormalized with respect to their maximum values. Generally, conventionalsystem 100 stops working at about a half wave of defocus.

FIGS. 7-9 are gray images of a grid viewed by optical system 200 or 300with extended depth of field at best focus, five waves of defocus, andeight waves of defocus, respectively. The WFC phase function used in allsimulations is a cubic function with 6 waves of total phase deviation,and it is mathematically given byP(x,y)=1.5(x ³ +y ³)  Equation (8)where x and y are normalized to a range of −1 to 1 across the extent ofthe function (e.g., across a width of any of WFC elements 208, 308 or314), with x=y=0 corresponding to the optical axis.

Specifically, FIG. 7 shows gray image 700 including grid lines 702, FIG.8 shows gray image 800 including grid lines 802, and FIG. 9 shows grayimage 900 including grid lines 902. As can be appreciated by comparingFIG. 4 to FIG. 7, system 200 or 300 does not image grid lines as clearlyas system 100 at best focus. However, system 200 or 300 isadvantageously able to image grid lines clearly at larger amounts ofdefocus than system 100. For example, as shown in FIGS. 8 and 9, system200 or 300 can image grid lines clearly at a defocus of five or eightwaves, while system 100 is unable to image grid lines at these amountsof defocus, as shown in FIGS. 5 and 6 where grid lines are notdistinguishable.

FIGS. 10-12 are simulated contour plots of gray images 400, 500, and 600by conventional system 100. As shown in FIG. 10, contour plot 1000 byconventional system 100 clearly reveals grid lines 1002 at best focus.However, contour plot 1100 at five waves of defocus and contour plot1200 at eight waves of defocus do not show any grid lines, asillustrated in FIGS. 11 and 12, respectively. In other words, grid lineimage quality of conventional system 100 is sensitive to the depth offield.

FIGS. 13-15 are simulated contour plots 1300, 1400, and 1500 of grayimages 700, 800 and 900, respectively. Thus, contour plot 1300represents an image generated by system 200 or 300 at best focus,contour plot 1400 represents an image generated by system 200 or 300 atfive waves of defocus, and contour plot 1500 represents an imagegenerated by system 200 or 300 at eight waves of defocus. Grid lines1302, 1402, and 1502 are visible in each of contour plots 1300, 1400,and 1500, respectively. Images are normalized with respect to theirmaximum amplitude, and a total of ten contours are shown, in each ofcontour plots 1300, 1400, and 1500. As illustrated in FIGS. 13-15,optical system 200 or 300 with first WFC element 208 or 308 reveals gridlines at both best focus and defocus. In other words, system 200 or 300is less sensitive to the depth of field of grid mask 210 or 304 thanconventional system 100.

FIGS. 16-18 are simulated gray images 1600, 1700 and 1800 of a singlepoint imaged by conventional system 100 at best focus, five waves ofdefocus, and eight waves of defocus respectively. These imagesillustrate substituting a single point mask for the grid mask insimulations of conventional system 100, such that a point spreadfunction (PSF) of the projector 102 of system 100 is obtained. As shownin FIG. 16, image 1600 reveals nearly a single point 1602 at best focus.However, images 1700 and 1800 at defocus are no longer confined to apoint. Instead, image 1700 reveals a circular spot 1702 and thebrightest spot 1704 in the center of circular spot 1702, and image 1800reveals a large circular spot 1802 and the brightest spot 1804 in thecenter of circular spot 1802. Circular spot 1802 is larger than circularspot 1702.

FIGS. 19-21 are simulated gray patterns 1900, 2000, 2100 of a pointimaged by optical system 200 or 300 at best focus, five waves ofdefocus, and eight waves of defocus, respectively. These figuresillustrate substituting a single point mask for grid mask 304 insimulations, such that a point spread function (PSF) of the projectorarm 320 of imaging system 300 is obtained. Gray pattern 1900 (FIG. 19)includes dots 1904 in a substantially triangular shape with a rightangle, and reveals a darkest point 1902 at a corner of the right angleof the triangular shape. The darkest spot 1902 is actually the brightestspot in a real image because of the inverted gray scale in FIG. 19.

The triangular shape is a result of the extended phase modulation alongthe horizontal and vertical axes because of the use of the cubicfunction. If a circularly symmetric phase function had been used, forexample, the resulting PSF would also be circularly symmetric.

The lateral shift arises as a result of the superposition of a secondorder phase deviation (defocus) over a third order phase deviation (thecubic phase function) of opposite signs, resulting in a linear phaseteen with slope that varies as a function of defocus. The Fouriertransform of a linear phase slope is a spatial shift proportional to theslope and, therefore, proportional to the amount of defocus. The amountof defocus, on the other hand, is proportional to the distance between agiven point at the scene and the plane of best focus.

Gray pattern 2000 of FIG. 20 includes dots 2004 mainly along the x axisand y axis and the darkest spot 2002 near the intersection of the x axisand the y axis. Gray pattern 2100 of FIG. 21 includes dark stripes 2106near the x axis and the y axis. Gray pattern 2100 also includes lighterstripes 2104 away from the x axis and the y axis. Images use invertedcolor scale for clarity (that is, dark spots represent points ofstronger light brightness). Thus, a “light stripe” would actually haveless light than a “dark stripe”. The darkest spot 2102 is near theintersection of the x axis and the y axis. Thus, spots 2002 and 2102 ofgray patterns 2000 and 2100 viewed in system 200 or 300 appear to bedarker and far more compact than the respective spots 1702 and 1802 seenin system 100, indicating a better ability to detect a single point atdefocus.

The differences between small spot 1602 at best focus and largercircular spots 1702 and 1802 at defocus (FIGS. 16-18) are much largerthan the differences between darkest spot 1902 at best focus and darkestspots 2002 and 2102 at defocus. In other words, system 200 or 300 candetect a single point better than conventional system 100 at variousamounts of defocus, which can also be illustrated by comparing PSFs ofprojector 102 of conventional optical system 100 with PSFs of projectionarm 220 or 320 of system 200 or 300. Imaging systems are usually assumedto be linear, in which case superposition applies. That is, any imagecan be decomposed as a summation of points, and the ability toaccurately detect or project a single point indicates the ability toaccurately detect or project the multiple points that form any givenimage.

FIGS. 22-24 are three-dimensional graphical representations of PSFs2200, 2300, and 2400 of projector 102 of conventional system 100 at bestfocus, five waves of defocus, and eight waves of defocus, respectively.PSF 2200 at best focus is quite different from PSF 2300 at five waves ofdefocus and PSF 2400 at eight waves of defocus. Major peaks 2202, 2302and 2402 correspond to darkest or brightest spots 1602, 1704 and 1804 ofFIGS. 16, 17, and 18, respectively. The darkest spot 1602, would be thebrightest spot in a real image, because the gray scale is inverted tomake the images easier to visualize in a white (paper) background. Thelarge differences among the PSFs suggest that it is difficult to detecta point by conventional system 100 at five waves of defocus or eightwaves of defocus.

FIGS. 25-27 are three-dimensional graphical representations of PSFs2500, 2600, and 2700 of projection arm 220 or 320 of system 200 or 300with first WFC element 208 or 308 at best focus, five waves of defocus,and eight waves of defocus respectively. Referring to FIG. 25, PSF 2500at best focus includes a major peak 2502 and small peaks 2504 near themajor peak 2502. Referring to FIG. 26, PSF 2600 at five waves of defocusincludes a major peak 2602 and weak peaks 2604 near major peak 2602.Referring to FIG. 27, PSF 2700 at eight waves of defocus still includesa major peak 2702 and weak peaks 2704 near major peak 2702. Weak peaks2704 at eight waves of defocus appear to be more significant than weakpeaks 2604 at five waves of defocus. The major peaks 2502, 2602 and 2702correspond to the darkest spots 1902, 2002 and 2102 of FIGS. 19, 20, and21 respectively. Regardless of focus or defocus, major peaks 2502, 2602,and 2702 can be detected by optical system 200 or 300 with the first WFCelement 208 or 308. In other words, in system 200 or 300, PSF 2500 atbest focus is similar to PSFs 2600 and 2700 at defocus. In contrast, insystem 100, PSF 2200 at best focus is very different from PSFs 2300, and2400 at defocus. More specifically, the PSFs of system 200 or 300 do notchange much at a large amount of defocus, even at eight waves ofdefocus.

FIGS. 28-30 are simulated contour plots 2800, 2900, and 3000 of grayimages 1600, 1700, and 1800 by conventional system 100 at best focus,five waves of defocus, and eight waves of defocus, respectively. FIGS.28-30 are normalized with respect to their individual maximum amplitude,and ten contour levels were used in each case. Also, contour plots 2800,2900, and 3000 are zoomed around the largest contour level so thatdetail is clearly shown.

FIGS. 31-33 are simulated contour plots 3100, 3200 and 3300 of grayimages 1900, 2000 and 2100, respectively, by optical system 200 or 300.As illustrated in FIG. 31, contour plot 3100 at best focus includes amajor peak 3102 that has the largest number of closed loops and is nearthe intersection of the x axis and the y axis. Contour plot 3100 alsoincludes minor peaks 3104 that have fewer closed loops and are away fromthe x axis and the y axis, and the intersection of the x axis and the yaxis. The darkest spot 1902 of image 1900 corresponds to major peak 3102of contour plot 3100. The darkest spot 1902 is actually the brightestspot in a real image because of the inverted gray scale in FIG. 19.

Referring to FIG. 32, contour plot 3200 at five waves of defocusincludes a major peak 3202 that has the largest number of closed loopsnear the intersection of the x axis and the y axis. Contour plot 3200also includes axial peaks 3204 that have elongated loops and are alongthe x axis and the y axis, and minor peaks 3206 that have a smallnumbers of closed loops and are away from the x axis and the y axis andthe intersection of the x axis and the y axis.

Referring to FIG. 33, contour plot 3300 at eight waves of defocus issimilar to contour plot 3100 at best focus and contour plot 3200 at fivewaves of defocus. Contour plot 3300 includes a major peak 3302 that hasthe largest number of closed loops near the intersection of the x axisand the y axis. However, minor peaks 3304 of contour plot 3300 havefewer loops and are more spread away from the x axis and the y axis thanminor peaks 3104 and 3206 of contour plots 3100 and 3200, respectively.Note that in FIGS. 31-33, each image is normalized with respect to itsrespective maximum amplitude, and each plot is zoomed around the largestcontour levels in order to show detail. Also note that the scaledifferences among FIGS. 31-33 for system 200 or 300 with extended depthof field are much smaller than the scale difference among FIGS. 28-30for conventional system 100. The reason for this difference is thelarger PSF defocus invariance or extended depth of field provided byimaging systems 200 or 300 with first WFC element 208 or 308.

Additional examples of optical systems with extended depth of field areprovided below. FIG. 34 illustrates an alternative embodiment 3400 ofoptical system 300. System 3400 includes projection arm 3420 anddetection arm 330. Projection arm 3420 includes a light source 3402, anoptical lens 3404 and a physical medium 3406. Lens 3404 is placedapproximately one focal length from light source 3402, condenses lightfrom light source 3402, and provides substantially uniform illuminationfor physical medium 3406. Projection arm 3420 also includes beamsplitter 310 to allow dual use of lens 312, as described for system 300.Physical medium 3406 is placed approximately one focal length from dualpurpose lens 312. A calculated computer generated hologram (CGH) istransferred to physical medium 3406. The CGH contains a first computerrepresentation of a grid mask (not shown) and a second computerrepresentation of the Fourier transform of first WFC element 308. Thedetection arm 330 and other components in system 3400 are the same asfor system 300.

Physical medium 3406 may be an optically transparent plastic, such aspoly(methyl methacrylate) (PMMA) that is embossed with a transparentpattern that modulates transmitted light with the phase variationsdetermined by the CGH algorithm. PMMA allows creation of a number ofCGHs at relatively low cost. An advantage of system 3400 over system 300is that grid mask 304, lens 306B, and first WFC element 308 of FIG. 3 donot need to be physically present. However, light source 3402 forilluminating embossed physical medium 3406 needs to provide at leastpartially coherent light. Light source 3402 may be a laser diode toprovide a relatively high signal-to-noise ratio and efficiency amonglight sources. Light source 3402 may also be an LED, to providesatisfactory results at relatively lower cost than a laser diode.

The CGH may be generated by using a computer algorithm. A computerrepresentation of a desired image is first generated by the computeralgorithm, including a first computer representation of grid mask 304and a second computer representation of first WFC element 308. Next, acomputer algorithm, such as Beamprop, calculates an optical wavefront ata plane where the CGH is physically located. Typically, a gridprojection takes place at a Fourier plane of the CGH. The CGH may becalculated and provided in a form for fabrication of physical medium3406, for example, in the form of bars of varying heights at variouspositions, or a surface relief pattern, through using a CGHprescription, such as a Lohmann hologram. An initial CGH may be providedby a Fourier transform of the desired image. Then, an intermediate CGHmay be calculated by using the computer algorithm to obtain an errorimage, which is a difference between the desired image and theintermediate image. By using optimization techniques, such as gradientdescent or weight perturbation, the intermediate CGH may be modifieduntil a final CGH obtains a minimal error in the error image. Moredetails about CGH are provided in “Complex Spatial Filtering with BinaryMasks” by B. R. Brown and A. W. Lohmann, Applied Optics, Vol. 5, Issue6, pp. 967-969 (1966).

It will be appreciated by those skilled in the art that optical system200 or 300 may have various configurations. In a particular embodiment,beam splitter 310 may be movable or removable, such as to reduce loss oflight during conventional image capture. Beam splitter 310 is not 100%reflective, and at least some light projected by beam splitter 310toward object 318 will be lost.

In three-dimensional image capture, a volumetric resolution of animaging system is limited by the system's ability to distinguish betweenindividual points in object space and accurately determine theirposition in three-dimensional space. The object space is the physicalspace around an object. In contrast, image space is the space around theimage of the object. For example, in FIG. 3, the object space forprojector aim 320 would be around grid mask 304, while the object spacefor detector arm 330 would be around object 318. Object 318 is in imagespace for projector arm 320, but in object space for detector arm 330.Detector 316 is in image space for detector arm 330.

Individual points in the object space can be determined by the “centerof mass” of the crossing point of grid images, that is, by accuratelycorrelating grid line centroids to their respective grid crossings. Thepoint spread function of projection arm 220 or 320 of system 200 or 300changes substantially less with defocus than the PSF of projector 102 ofconventional system 100. Moreover, first WFC element 208 or 308 providesa significant increase in depth of field with a minimum loss inmodulations of intensity. Consequently, the grid line centroids may bemore clearly identified after defocus, resulting in a substantialincrease in volumetric or spatial resolution, at a much larger DOF.Structural illumination increases volumetric resolution or spatialresolution by collecting information over a larger volume in objectspace. By increasing the DOF, the volumetric resolution is increased.

Having described several embodiments, it will be recognized by thoseskilled in the art that various modifications, alternative constructionsand equivalents may be used without departing from the spirit of thedisclosure. Additionally, a number of well known mathematicalderivations and expressions, processes and elements have not beendescribed in order to avoid unnecessarily obscuring the presentdisclosure. Accordingly, the above description should not be taken aslimiting the scope of the disclosure.

It should thus be noted that the matter contained in the abovedescription or shown in the accompanying drawings should be interpretedas illustrative and not in a limiting sense. The following claims areintended to cover generic and specific features described herein, aswell as all statements of the scope of the present method and system.

What is claimed is:
 1. An optical system for capturing images of athree-dimensional object, the optical system comprising a projector forstructured illumination of the object, wherein the projector comprises:a light source; a grid mask positioned between the light source and theobject for structured illumination of the object; a first WavefrontCoding (WFC) element having a phase modulating mask positioned betweenthe grid mask and the object to receive patterned light from the lightsource through the grid mask, wherein the first WFC element isconstructed and arranged such that a point spread function of theprojector is substantially invariant over a wider range of depth offield of the grid mask than a point spread function of the projectorwithout the first WFC element; a detector for detecting the object withstructured illumination from the projector; and a second WFC elementlocated between the object and the detector such that a first image ofthe object and a second image of the grid mask on the detector have alarger depth of field than without the second WFC element; wherein thesecond WFC element comprises a circularly symmetric WFC elementexpressed as P(ρ)=f(ρ), wherein ρ²=x²+y², ρ is a radius of a Fourierplane of the detector, and f(ρ) is a high order polynomial.
 2. Theoptical system of claim 1, wherein the depth of field is at least halfof a wave of defocus relative to a center wavelength of the lightsource.
 3. The optical system of claim 1, wherein the first WFC elementis configured to concentrate at least 60% of phase modulations alonggrid lines of the grid mask.
 4. The optical system of claim 1, whereineach of the first WFC element and second WFC element comprises acircularly symmetric WFC element.
 5. The optical system of claim 1,wherein the second WFC element is configured to be movable or removable.6. The optical system of claim 1, wherein the grid mask and the firstWFC element are part of a single component, the single componentcomprising a molded plastic.
 7. The optical system of claim 1, whereinlight from the light source comprises at least one of visible light andinfrared light.
 8. An optical system for capturing images of athree-dimensional object, the optical system comprising a projector forstructured illumination of the object, wherein the projector comprises:a light source; a grid mask positioned between the light source and theobject for structured illumination of the object; a first WavefrontCoding (WFC) element having a phase modulating mask positioned betweenthe grid mask and the object to receive patterned light from the lightsource through the grid mask, wherein the first WFC element isconstructed and arranged such that a point spread function of theprojector is substantially invariant over a wider range of depth offield of the grid mask than a point spread function of the projectorwithout the first WFC element; a detector for detecting the object withstructured illumination from the projector; and a second WFC elementlocated between the object and the detector such that a first image ofthe object and a second image of the grid mask on the detector have alarger depth of field than without the second WFC element; wherein eachof the first and second WFC elements comprises a weakly non-separablefunction is expressed as${{P\left( {x,y} \right)} = {{\sum\limits_{i}^{N}{\sum\limits_{j}^{N}{\alpha_{1}x^{i}}}} + {\beta_{j}y^{j}} + {\gamma\; x^{i}y^{j}}}},$ the weakly non-separable function for the second WFC element isexpressed as${{P\left( {X^{\prime},Y^{\prime}} \right)} = {{\sum\limits_{i}^{N}{\sum\limits_{j}^{N}{\alpha_{i}\left( {- X^{\prime}} \right)}^{i}}} + {\beta_{j}\left( {- Y^{\prime}} \right)}^{j} + {{\gamma\left( {- X^{\prime}} \right)}^{i}\left( {- Y^{\prime}} \right)^{j}}}},$ i and j are positive integers, wherein Y′ axis is in the same directionas y axis, Z′ axis is in the same direction as x axis, and X′ axis is inthe opposite direction of z axis.
 9. An optical system for capturingimages of a three-dimensional object, the optical system comprising aprojector for structured illumination of the object, wherein theprojector comprises: a light source; a grid mask positioned between thelight source and the object for structured illumination of the object; afirst Wavefront Coding (WFC) element having a phase modulating maskpositioned between the grid mask and the object to receive patternedlight from the light source through the grid mask, wherein the first WFCelement is constructed and arranged such that a point spread function ofthe projector is substantially invariant over a wider range of depth offield of the grid mask than a point spread function of the projectorwithout the first WFC element; a detector for detecting the object withstructured illumination from the projector; and a second WFC elementlocated between the object and the detector such that a first image ofthe object and a second image of the grid mask on the detector have alarger depth of field than without the second WFC element; wherein eachof the first and second WFC elements comprises a high order separablefunction, and wherein the high order separable function for the firstWFC element is expressed as${{P\left( {x,y} \right)} = {{\sum\limits_{i}^{N}{\sum\limits_{j}^{N}{\alpha_{i}x^{i}}}} + {\beta_{j}y^{j}}}},$ and the high order separable function for the second WFC element isexpressed as${{P\left( {X^{\prime},Y^{\prime}} \right)} = {{\sum\limits_{i}^{N}{\sum\limits_{j}^{N}{\alpha_{i}\left( {- X^{\prime}} \right)}^{i}}} + {\beta_{j}\left( {- Y^{\prime}} \right)}^{j}}},$ wherein Y′ axis is in the same direction as y axis, Z′ axis is in thesame direction as x axis, and X′ axis is in the opposite direction of zaxis.
 10. An optical system for capturing images of a three-dimensionalobject, the optical system comprising a projector for structuredillumination of the object, wherein the projector cormprises: a lightsource; a grid mask positioned between the light source and the objectfor structured illumination of the object; a first Wavefront Coding(WFC) element having a phase modulating mask positioned between the gridmask and the object to receive patterned light from the light sourcethrough the grid mask, wherein the first WFC element is constructed andarranged such that a point spread function of the projector issubstantially invariant over a wider range of depth of field of the gridmask than a point spread function of the projector without the first WFCelement; a detector for detecting the object with structuredillumination from the projector; and, a second WFC element locatedbetween the object and the detector such that a first image of theobject and a second image of the grid mask on the detector have a largerdepth of field than without the second WFC element; wherein each of thefirst and second WFC elements comprises a cubic function, and whereinthe cubic function for the first WFC element is expressed as αx³+βy³,and the cubic function for the second WFC element is expressed as−(αX′³+βY′³), wherein Y′ axis is in the same direction as y axis, Z′axis is in the same direction as x axis, and X′ axis is in the oppositedirection of z axis.